Technical Field
This disclosure relates generally to machine learning algorithms and, in particular, to algorithms for use in training Support Vector Machines (SVMs).
Background of the Related Art
The Support Vector Machine (SVM) is one of the most robust machine-learning algorithms developed to date, and it is one of the most commonly-used ones. Over the last 15 years, algorithms for training SVMs have been evolving from being scalable to thousands of examples, to being scalable to millions. Nevertheless, current state-of-the-art sequential linear SVM solvers are relatively slow if they are trained over tens of millions of high-dimensional examples. The scalability of the state-of-the-art sequential solvers of nonlinear SVM is even worse.
It is also known to perform distributed SVM training. Prior attempts in distributed SVM training have focused on parallelizing particular steps of sequential solvers. Most of these approaches focused on parallelizing the solvers of nonlinear SVMs. Attempts to parallelize the solvers of linear SVMs have required keeping the entire dataset in the memory of a master node. This constraint limits the size of the training set to one that will fit into memory.